Image object classification method, system and computer readable medium

ABSTRACT

An image object classification method and system are disclosed. The method is executed by a processor coupled to a memory. The method includes: providing an image file including at least one image object, performing a process of extracting multiple binary-classified characteristics on the image object to obtain a plurality of first results independent of each other in categories, combining the plurality of first results in a manner of dimensionality reduction based on concatenation, performing a process of characteristics abstraction on the combined first results to obtain a second result, and performing a process of characteristics integration on the plurality of first results and the second result in a manner of dot product of matrices to obtain a classification result.

FIELD OF INVENTION

The present disclosure relates to a classification technology and,specifically, to an image object classification method and system.

BACKGROUND OF INVENTION

The development and applications of machine learning have graduallybecome an important field of research. A large amount of data (orinformation) can be used to train machine learning models, and thetrained models can be used to obtain certain prediction information.

Object classification technology is widely used, such as being appliedto image object classification. The conventional object classificationmodel usually emphasizes that objects can be classified as a singlecategory of information as an output result.

However, there are still shortcomings in classifying objects into asingle category. Taking, as an example, a digital file containingmultiple types of images, the actual application scenario needs toclassify an object into multiple categories, but this is a factor thatis usually not considered in conventional classification. Although therehave been some classification techniques, they are still not suitablefor certain applications.

In light of this, it is necessary to provide a technical solutiondifferent from the past to solve problems with the prior art.

SUMMARY OF INVENTION

One objective of the present disclosure is to provide an image objectclassification method that can classify image objects into multiplecategories and is favorably suitable for digital files containingmultiple image categories.

Another objective of the present disclosure is to provide an imageobject classification system that can classify image objects intomultiple categories and is favorably suitable for digital filescontaining multiple image categories.

Another objective of the present disclosure is to provide a tangible,non-transitory, computer readable medium that can classify image objectsinto multiple categories and is favorably suitable for digital filescontaining multiple image categories.

To achieve the above objective, one aspect of the present disclosureprovides an image object classification method, executed by a processorcoupled to a memory, including: providing an image file including atleast one image object; performing a process of extracting multiplebinary-classified characteristics on the image object to obtain aplurality of first results independent of each other in categories;combining the plurality of first results in a manner of dimensionalityreduction based on concatenation, and performing a process ofcharacteristics abstraction on the combined first results to obtain asecond result; and performing a process of characteristics integrationon the plurality of first results and the second result in a manner ofdot product of matrices to obtain a classification result.

In one embodiment of the present disclosure, the classification resultincludes the second result, the second result further includesclassification reliabilities of the plurality of first results, whereinafter the classification result is obtained, the method further includesdocumenting the classification result, which comprising: selecting atleast one of the plurality of first results according to a result ofsorting the classification reliabilities of the plurality of firstresults; and recording at least one category attribution name, at leastone object position, and at least one object size corresponding to theat least one selected first result, in a file.

In one embodiment of the present disclosure, the object positionincludes a combination of a start-point coordinate and an end-pointcoordinate of the image object, or a combination of a center coordinate,an object length, and an object width of the image object.

In one embodiment of the present disclosure, after the classificationresult is obtained, the method further includes performing agraphic-text process on the classification result to present theclassification result in the image file in the form of a graphic block,a text block, or a combination thereof.

In one embodiment of the present disclosure, the second result isobtained by performing the process of characteristics abstraction on thecombined first results in a fully connected manner.

In one embodiment of the present disclosure, the second result isobtained by performing the process of characteristics abstraction on thecombined first results through multi-layer perception.

In one embodiment of the present disclosure, the plurality of firstresults are sequentially concatenated into a combineddimensionality-reduced result.

In one embodiment of the present disclosure, the multiplebinary-classified characteristics are extracted on the image object froma plurality of image categories that include a plurality of schematicdiagrams of characteristics of an electronic component.

Another aspect of the present disclosure provides an image objectclassification system including a processor coupled to a memory storingat least one instruction configured to be executed by the processor toperform the above method.

Another aspect of the present disclosure provides a tangible,non-transitory, computer readable medium storing instructions that causea computer to execute the above method.

The image object classification method, system, and tangiblenon-transitory computer readable medium of the present disclosure areprovided for providing an image file including at least one imageobject; performing a process of extracting multiple binary-classifiedcharacteristics on the image object to obtain a plurality of firstresults independent of each other in categories; combining the pluralityof first results in a manner of dimensionality reduction based onconcatenation, and performing a process of characteristics abstractionon the combined first results to obtain a second result; and performinga process of characteristics integration on the plurality of firstresults and the second result in a manner of dot product of matrices toobtain a classification result. Thus, after the object is processed asmentioned above classification process, it is possible to outputimplicit information that the image object belongs to multiplecategories, which is beneficial to digital documents containing multipleimage categories.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic diagram of an image object classification systemaccording to some embodiments of the present disclosure.

FIG. 2 is a flowchart of an image object classification method accordingto some embodiments of the present disclosure.

FIG. 3 is a schematic diagram of a multi-classifier used in someembodiments of the present disclosure.

FIG. 4 is a schematic diagram of another non-multi-classifier not usedin some embodiments of the present disclosure.

FIG. 5A is a schematic diagram of a first image example used to classifya single image object in some embodiments of the present disclosure.

FIG. 5B is a schematic diagram of a second image example used toclassify a single image object in some embodiments of the presentdisclosure.

FIG. 5C is a schematic diagram of a third image example used to classifya single image object in some embodiments of the present disclosure.

FIG. 5D is a schematic diagram of a fourth image example used toclassify a single image object in some embodiments of the presentdisclosure.

FIG. 5E is a schematic diagram of a fifth image example used to classifya single image object in some embodiments of the present disclosure.

FIG. 6A is a schematic diagram of a first image example used to classifymultiple image objects in some embodiments of the present disclosure.

FIG. 6B is a schematic diagram of a second image example used toclassify multiple image objects in some embodiments of the presentdisclosure.

DETAILED DESCRIPTION OF EMBODIMENTS

The following description of the various embodiments is provided toillustrate the specific embodiments of the present disclosure.Furthermore, directional terms mentioned in the present disclosure, suchas upper, lower, top, bottom, front, rear, left, right, inner, outer,side, surrounding, central, horizontal, lateral, vertical, longitudinal,axial, radial, uppermost, and lowermost, which only refer to thedirection of drawings. Therefore, the directional terms configured asabove are for illustration and understanding of the present disclosureand are not intended to limit the present disclosure.

Please refer to FIG. 1 . One aspect of the present disclosure providesan image object classification system, which can be configured toinclude a processing device 1 including a classification module 11 and aformat conversion module 12 coupled to each other. The coupling mannercan be coupling or connection methods, such as wired connection,wireless transmission, or data exchange. The classification module 11and the format conversion module 12 can be software modules, hardwaremodules, or software and hardware cooperative operation modules. Forexample, the classification module 11 can be configured to have datainput, processing, and output functions, such as data reading,calculation, and display, to generate a classification result R based onexternal data D. The format conversion module 12 can be configured tohave a data format conversion function. Furthermore, the image objectclassification system can further include a database (not shown). Thedatabase can be coupled to the classification module 11 and the formatconversion module 12 to store relevant data for processing, such as datasets or parameters for training machine learning modules. The imageobject classification system may further include other parts, such as acomponent classification unit A1 and a component view recognition unitA2. The component classification unit A1 is used for performing acomponent classification operation. The component view recognition unitA2 is used for a component view direction recognition operationaccording to the classification result R.

For example, the image object classification system may be configured toinclude a processor and a memory coupled to the processor, wherein thememory stores at least one instruction executed by the processor toperform an image object classification method provided by another aspectof the present disclosure, which is illustrated later, but not limitedhere.

Another aspect of the present disclosure that provides an image objectclassification method that is executed by a processor coupled to amemory and includes providing an image file, e.g., being converted froma document file, wherein the image file includes at least one imageobject; performing a process of extracting multiple binary-classifiedcharacteristics on the image object to obtain a plurality of firstresults independent of each other in categories; combining the pluralityof first results in a manner of dimensionality reduction based onconcatenation, and performing a process of characteristics abstractionon the combined first results to obtain a second result; and performinga process of characteristics integration on the plurality of firstresults and the second result in a manner of dot product of matrices toobtain a classification result. Embodiments that can be implemented aregiven later as examples to provide an understanding of relevant content,but not limited here.

For example, as shown in FIG. 2 , the image object classification methodmay include steps S1 to S6. At least one part of these steps may beappropriately changed, simplified, or omitted based on actualapplications to complete at least one part of the above-mentionedembodiment.

In step S1, may be configured to input external data as a basis forsubsequent image object classification. For example, the external datais read. The external data can be an image file. As an example, theimage file can be provided after a document file is converted and storedby an external machine, but not limited here.

The external data can also be the document file, and the processor canconvert the document file to provide the image file. The image file isat least provided in the manner mentioned above and is not restrictedhere. For example, the document file can be a digital document filecontaining various graphic examples (such as different electroniccomponent views) and text examples (such as explanatory text). Forexample, “*.pdf,” a portable document format, e.g., the file's contentcan include technical documents (datasheets) related to electroniccomponents, but not limited here.

The digital document file can also be in other file formats, e.g., otherfile formats, including, but is not limited to, graphics and text, suchas “*.doc” or “*.odt.” Additionally, the external data can also includeother data, such as tables. The image file can be in compressed oruncompressed file format, such as “*.jpg,” but it is not limited here.The image file can also be in other file formats, such as “*.png” or“*.bmp.”

The image files described later can refer to file data or screen contentof them displayed on a display device, which can also be called aspictures. Subsequently, step S2 can be performed.

In step S2, may be configured to perform a process of extractingmultiple binary-classified characteristics on at least one image objectto obtain a plurality of first results independent of each other incategories. For example, the image file containing at least one imageobject is simultaneously sent into a plurality of modules with differentbinary classifier functions to obtain a plurality of first results.

The plurality of first results are independent of each other in statesof classification. A setting manner to features (such as architectureand parameters) of the module can be understandable by a person havingordinary skill in the art, and will not be repeated here. Subsequently,a step S3 can be performed.

For example, image data in a data set can be input into a plurality ofbinary classification modules that have been trained for differentfeatures to perform operations to generate a plurality of output resultsas the plurality of first results. For example, the modules with binaryclassifier functions can perform feature extraction for different imageobject features. An application scenario of electronic components istaken as an example. Schematic diagrams of various features ofelectronic components can be classified, such as appearance features,electrical features, and application features.

For example, drawing views of electronic components may be adopted. Eachof drawing-view diagrams (such as top-view, bottom-view, and side-view)has line segments and geometries for showing the components' size. Inaddition, pin-assignment diagrams (also called such as pinconfigurations, pinout diagrams, pin function diagrams etc., e.g.,integrated circuit (IC) diagrams showing the feature of pins,hereinafter referred to as IC diagrams) of the electronic components maybe adopted. Each of the pin-assignment diagrams of the IC package hasgeometries showing pins and text.

For example, a circuit diagram may be adopted (For example, the circuitformed by connecting symbols of electronic components has geometricgraphs and lines). Also, characteristic curve diagrams may be adopted(For example, characteristic curve diagrams of the voltage or current ofelectronic components have waveform information formed by continuouslyextending lines).

For example, signal timing diagrams may be adopted (For example, each ofthe clock's timing diagrams, input, and output signals of electroniccomponents has a closed block waveform composed of continuouslyextending lines used to present a continuous waveform relationship. Themain difference between signal timing diagrams and the characteristiccurve diagrams is there are more closed blocks and turning lines), butnot limited here.

In addition, other image object classification scenarios, such as amobile phone manual, may include a schematic diagram of appearancefunctions and a schematic diagram of screen functions. Graphiccharacteristics of the mobile phone manual can also be analyzed as abasis for classifying image objects.

In step S3, may be configured to combine the plurality of first resultsin a manner of dimensionality reduction based on a concatenation. Thedata of the plurality of first results are sequentially concatenated.The data arranged on the plane is reduced in dimensionality andconverted into linearly arranged data. A process of characteristicsabstraction can be performed on the combined first results to obtain asecond result.

For example, the plurality of first results are processed in a combineddimensionality reduction process (such as converting two-dimensionaldata into one-dimensional data). For example, the plurality of firstresults are sequentially concatenated to form a combineddimensionality-reduced result. Then, the combined dimensionality-reducedresult is processed in a feature abstraction process.

For example, the features (patterns) of the combineddimensionality-reduced result can be summarized as an essentialrepresentation with less information, and irrelevant details are ignoredto reduce complexity. It becomes a model that a device can efficientlyprocess with computing capabilities, such as a multi-layer perceptronmachine (also known as a fully connected layer).

For example, a one by one (1×1) convolution can be used for dimensionalconversion. A dilated convolution can be used to adjust a receptivefield to increase or decrease the extracted feature amount to obtain thesecond result. For example, the second result includes the symbolicvalue of the plurality of first results, such as the classificationreliabilities of the plurality of first results.

The value ranges of all categories are independent and between 0 and 1,Sigmoid[0:1]. The setting method is understandable by those skilled inthe art and will not be repeated. Subsequently, a step S4 can beperformed.

In step S4, may be configured to perform a feature integration processon the plurality of first results and the second result in a manner ofdot product of matrices to obtain a classification result. For example,the second result is sent to a model with a feature classifier function,such as a neural network module with a fully connected layer. The fullyconnected layer used in the embodiment of the present disclosure andanother one that uses one by one (1×1) convolution is different in theoverall application scenario.

For example, as shown in FIG. 3 , in this example, the plurality offirst results obtained by the binary classifier are simultaneouslyprovided to the fully connected layer used here as the input data andprovided to a feature integrator as operation source information. Forexample, the second result output by the fully connected layer and theplurality of first results will be processed in a combining operation,such as a superposition operation. For example, an operation of dotproduct of matrices can be adopted. For example, a product obtained bymultiplying elements corresponding to positions in two matrices isregarded as an element of the other matrix at the same positions.

The operation result of the feature integrator includes a lot oforiginal binary classification information and its derivativesextraction of classification information to combine differentclassification information as the classification result. The modulefeatures (such as architecture and parameters) are set in a manner thatcan be understandable by those skilled in the art and will not berepeated here. Subsequently, step S5 and/or step S6 can be performed.

Optionally, in step S5, the classification result includes the secondresult, and the second result further includes the classificationreliabilities of the plurality of first results. Therefore, after theclassification result is obtained, the classification result can bedocumented. At least one of the plurality of first results can beselected from a sorted result of the plurality of first results torecord at least one category attribution name, at least one objectposition, and at least one object size. For example, when theclassification result is obtained, the classification result includesinformation in the second result.

The second result contains the classification reliabilitiescorresponding to the plurality of first results. Therefore, theplurality of first results can be sorted according to the classificationreliabilities. For example, the classification reliabilities of theplurality of first results can be normalized, such as using Max-Minnormalization, L1 normalization, and L2 normalization, which areunderstandable by those skilled in the art and will not be repeatedhere. Subsequently, these normalized reliabilities can be sorted.

According to the sorted result, the corresponding category attributionname, object position, and object size of the plurality of first resultsare recorded. Subsequently, the category attribution name, objectposition, and object size corresponding to at least one of the pluralityof first results can be selected according to preset conditions tocreate a document.

For example, one of the highest or top few reliabilities can be selectedto save the data storage space and data transmission volume. Forexample, the position of the object includes any one combination of thefollowing: a combination of a starting-point coordinate (such as theupper left corner coordinate) and an ending-point coordinate (such asthe lower right corner coordinate) of the image object, or a combinationof a center coordinate, a length, or a width of the image object, butnot limited to here.

Optionally, in step S6, after the classification result is obtained,which may be further processed to generate graphs and texts. Theclassification result is presented in the image file as graphic blocks,word blocks, or a combination thereof. For example, the classificationresult may be presented in graphs (such as frames with different colors)and texts (such as black text with different background colors).

For example, a color block may be pasted at the top of the block, and acategory of the classification result may be filled in a color blockwith words. When the graphic blocks belong to a plurality of categories,it is convenient to distinguish the categories with different colors toavoid confusion, but not limited here. For example, in another manner,the image file and the classification result can be separately stored infolders named with different category names. The same image file canappear in multiple folders at the same time, but not limited here.

Optionally, in one embodiment, the classification result includes thesecond result, the second result further includes classificationreliabilities of the plurality of first results, wherein after theclassification result is obtained, the method further includes:documenting the classification result, which including: selecting atleast one of the plurality of first results according to a result ofsorting the classification reliabilities of the plurality of firstresults; and recording at least one category attribution name, at leastone object position, and at least one object size corresponding to theat least one selected first result, in a file. In this way, theclassification result can be documented so that the classificationresult can be stored as field contents that are easy to understand tousers, which helps related analyses.

Optionally, in one embodiment, the object position includes acombination of a start-point coordinate and an end-point coordinate ofthe image object, or a combination of a center coordinate, an objectlength, and an object width of the image object. In this way, an objectposition format can be set as parameters based on coordinates,facilitating subsequent image processing (such as posting graphs).

Optionally, in one embodiment, after the classification result isobtained, the image object classification method further includes:performing a graphic-text process on the classification result topresent the classification result in the image file in the form of agraphic block, a text block, or a combination thereof. In this way, theclassification result can be processed to generate graphs and texts. Theclassification result can be directly presented in the image file, whichhelps the user to intuitively know the classification result.

Optionally, in one embodiment, the second result is obtained byperforming the process of characteristics abstraction on the combinedfirst results in a fully connected manner.

Optionally, in one embodiment, the second result is obtained byperforming the process of characteristics abstraction on the combinedfirst results through multi-layer perception. In this way, thecharacteristics of the plurality of first results are abstractive by themulti-layer perception manner. It is beneficial to simplify theprocessing complexity of intermediate data.

Optionally, in one embodiment, the plurality of first results aresequentially concatenated into a combined dimensionality-reduced result.In this way, all information contained in the plurality of first resultscan be aggregated to ensure data richness.

In addition, the amount of calculation required for the combineddimensionality-reduced processing can also be simplified, which isbeneficial to shorten the time acquiring the classification result.

Optionally, in one embodiment, the multiple binary-classifiedcharacteristics are extracted on the image object from a plurality ofimage categories that include a plurality of schematic diagrams ofcharacteristics of an electronic component. For example, drawing views,IC diagrams, circuit diagrams, characteristic curve diagrams, and signaltiming diagrams of the electronic component are discussed above, but notlimited here. In this way, different image categories, such as drawingviews, pin-assignment diagrams, circuit diagrams, characteristic curvediagrams, and signal timing diagrams of electronic components, can beeffectively classified. It can help to accelerate data interpretation,analysis, and related development schedule.

In another aspect, the present disclosure further provides an imageobject classification system, which includes a processor coupled to amemory storing at least one instruction configured to be executed by theprocessor to perform the above-mentioned method. The coupling manner canbe wired or wireless.

For example, the image object classification system can be configured asan electronic device with data processing functions. The electronicdevice can be cloud platform machines, servers, desktop computers,notebook computers, tablets, or smartphones, but not limited here.

The image object classification system, including a processor coupled toa memory storing at least one instruction configured to be executed bythe processor to perform a method including: providing an image fileincluding at least one image object; performing a process of extractingmultiple binary-classified characteristics on the image object to obtaina plurality of first results that are independent of each other incategories; combining the plurality of first results in a manner ofdimensionality reduction based on concatenation, and performing aprocess of characteristics abstraction on the combined first results toobtain a second result; and performing a process of characteristicsintegration on the plurality of first results and the second result in amanner of dot product of matrices to obtain a classification result. Theimplementation manner has been described above and will not be repeated.

In another aspect, the present disclosure further provides a tangible,non-transitory, computer readable medium, storing instructions thatcause a computer to execute operations including: providing an imagefile including at least one image object; performing a process ofextracting multiple binary-classified characteristics on the imageobject to obtain a plurality of first results that are independent ofeach other in categories; combining the plurality of first results in amanner of dimensionality reduction based on concatenation, andperforming a process of characteristics abstraction on the combinedfirst results to obtain a second result; and performing a process ofcharacteristics integration on the plurality of first results and thesecond result in a manner of dot product of matrices to obtain aclassification result.

After the instructions are loaded and executed by the computer, thecomputer can execute the aforementioned image object classificationmethod. For example, several program instructions can be implemented byusing existing programming languages to implement the aforementionedimage object classification methods, such as using Python in combinationwith Numpy, Matplotlib, and TENSORFLOW packages, but not limited here.

In another aspect, the present disclosure further provides acomputer-readable medium, such as an optical disc, a flash drive, or ahard disk, but is not limited here. It should be understandable that thecomputer-readable medium can further be configured as other forms ofcomputer data storage medium, e.g., cloud storage (such as ONEDRIVE,GOOGLE Drive, AZURE Blob, or a combination thereof), or data server, orvirtual machine. The computer can read the program instructions storedin the computer-readable medium. After the computer loads and executesthe program instructions, the computer can complete the above-mentionedimage object classification method.

To enable a person to understand the features of embodiments of thepresent disclosure, the following is an example of inputting a technicalfile (datasheet) of an electronic component as a document file andillustrates the classification process of the objects in the aboveembodiments. Still, it is not intended to be used as a limit.

It should be noted that the image object classification method, system,and tangible non-transitory computer readable medium of the aboveembodiments of the present disclosure mainly adopt the concepts of amulti-classifier X (as shown in FIG. 3 ). Binary multi-classificationfeature extraction processes are first processed on the input imageobject to obtain several first results that are independent of eachother in category. For example, the plurality of first results can becombined in dimensionality reduction based on concatenation andperforming a process of characteristics abstraction on the combinedfirst results to obtain a second result. The plurality of first resultsand the second result are combined in dot-multiplication in matrices toobtain a classification result (such as the value ranges of allcategories are independent and between 0 and 1, Sigmoid[0:1]).

In comparison, while another non-multi-classifier Y (as shown in FIG. 4) can be used for image classification, it mostly uses a module based ona convolutional neural network (CNN), e.g., models, such as visualgeometry group (VGG), deep residual network (ResNet), and real-timeobject detection (You only look once: Unified, Real-Time ObjectDetection, or Yolo, in short). The input image is convolved, pooled, andfully connected to obtain the output classification result (e.g., thesum of all categories is 1, Softmax[0:1]). However, after this modelprocesses a classification, only a single classification result can beobtained, which is different from the present disclosure.

In comparison, the image object classification method of theabove-mentioned embodiment of the present disclosure uses a plurality ofbinary classifiers. Each classifier classifies only two categories,e.g., when a request needs to be divided into five categories, tencategories can be used. The image object classification method of theabove-mentioned embodiment of the present disclosure has advantages:through each binary classification result, it can be learned which ofall categories are easy to distinguish and which of all categories arenot easy to distinguish.

In addition, a full connection is subsequently added to perform featureintegration training. For example, the classification results of severalcategories can be presented as multiple trust scores to be output (fivecategories or higher or lower), rather than just a single classificationresult.

In comparison, if the binary classifiers are used fornon-multi-classification tasks, then the binary classifiers' resultsshould be processed by a different decision-making mechanism. Thisdecision-making mechanism usually requires different thresholds todetermine a final result, resulting in more non-objective humanintervention.

Conversely, the image object classification method of theabove-mentioned embodiment of the present disclosure can use a fullyconnected layer instead of the decision-making mechanism. All decisionthresholds can be objectively obtained by learning, which caneffectively avoid effects of adjustments to the thresholds as a resultof artificial intervention.

TABLE l Input Data Table Label name quantity characteristic curve 1404circuit diagram 825 pin-assignment diagram 514 clock timing diagram 1011drawing-view diagram 1024 total 4778

For example, the image object classification method using themulti-classifier in the above-mentioned embodiment of the presentdisclosure will be compared with another method that uses thenon-multi-classifier. For example, the non-multi-classifier uses oneinput terminal, one convolutional neural network (CNN) module, and oneoutput terminal. The above-mentioned embodiment of the presentdisclosure uses one input terminal, ten binary classifiers, one fullyconnected network, and one output terminal. They are used for testinginput data (such as characteristic curve diagrams, circuit diagrams,pin-assignment diagrams, clock timing diagrams, and three-viewdiagrams), as shown in Table 1 above. The test results (such as modulename, sub-module, threshold, accuracy, and early-stop conditions) can beshown in Table 2 below.

TABLE 2 List of Test Results Early-stop Model Thre- Accu- ConditionsName Sub-model should racy Range Epoch The characteristic 0.5 87.53%<0.01 5 present curve-to- dis- circuit diagram closure characteristic0.5 94.84% <0.01 5 (Multi- curve-to-IC classi- diagram fication)characteristic 0.5 90.02% <0.005 5 curve-to-clock timing diagramcharacteristic 0.5 87.36% <0.005 5 curve-to- drawing- view diagramcircuit 0.5 97.46% <0.01 5 diagram-to- IC diagram circuit diagram- 0.591.37% <0.01 5 to-clock timing diagram circuit 0.5 86.10% <0.02 5diagram-to- drawing- view diagram IC diagram-to- 0.5 99.80% <0.001 5clock timing diagram IC diagram-to- 0.5 89.99% <0.01 5 drawing-viewdiagram clock timing 0.5 90.66% <0.005 5 diagram-to- drawing-viewdiagram binary-class-to- 0.5 82.60% no specific 5 multi-class stopconditions Non- CNN 0.5 79.52% <0.005 5 multi- classi- fication

It can be seen from Table 2, the classification performance of the imageobject classification method, system, and tangible non-transitorycomputer readable medium of the present disclosure adopting theaforementioned multi-classification are significantly better than theclassification performance of non-multi-classification.

The following examples illustrate the applications of embodiments of thepresent disclosure mentioned above as application examples forunderstanding the present disclosure's advantages, but is not limitedhere.

For example, as shown in FIG. 5A, which is an example of classifying asingle image object. An image file contains areas of characteristiccurve diagrams C1, C2, C3, C4, C5, and C6 for classifying image objects.In this example, many characteristic curve diagrams can be effectivelyclassified.

Further, as shown in FIG. 5B, which is another example of classifying asingle image object. An image file contains an area of circuit diagramK, an area of table B, and an area of text T1 for classifying imageobjects. In this example, the circuit diagram can be effectivelyclassified.

Further, as shown in FIG. 5C, which is another example of classifying asingle image object. An image file contains areas of IC diagrams V1, V2,V3, V4, and a text T2 for classifying image objects. In this example,the IC diagrams can be effectively classified.

Further, as shown in FIG. 5D, which is another example of classifying asingle image object. An image file contains an area of signal timingdiagram P for classifying image objects. In this example, the signaltiming diagram can effectively be classified.

Further, as shown in FIG. 5E, which is another example of classifying asingle image object. An image file contains many areas of drawing viewsof electronic components Va, Vb, Vc, Vd, Ve, an area of table Ba, and anarea of text Ta for classifying image objects. In this example, thedrawing views of electronic components can be effectively classified.

Therefore, the image object classification method, system, tangiblenon-transitory computer readable medium, and computer-readable medium ofthe present disclosure can indeed classify a single type of imageobjects distributed in a digital document file.

In addition, as shown in FIG. 6A, which is an example of classifyingmultiple image objects. An image file contains areas of drawing views ofelectronic components Vf, Vg, Vh, and a characteristic curve diagram Ca,combining with tables Bb, Bc, Bd, and text Tb, Tc, for classifying imageobjects. In this example, the categories of both the drawing views andthe characteristic curve diagrams can be effectively classified. Forexample, the classification order can be configured to find the drawingviews and then find the characteristic curve diagrams, but not limitedhere.

Further, as shown in FIG. 6B, which is an example of classifyingmultiple image objects. An image file contains an area of IC diagrams Viand areas of characteristic curve diagrams Cb and Cc combining withtables Be and Bf and an area of text Td for classifying image objects.In this example, the categories of both the IC diagrams and thecharacteristic curve diagrams can be effectively classified. Forexample, the classification order can be configured as first find the ICdiagrams, and then find the characteristic curve diagrams, but notlimited here. Therefore, the image object classification method, system,and tangible non-transitory computer readable medium of the presentdisclosure can indeed classify multiple image objects distributed in adigital document file.

Therefore, the image object classification method, system, and tangiblenon-transitory computer readable medium of the present disclosureperform the process of extracting multiple binary-classifiedcharacteristics, characteristics abstraction, and characteristicsintegration on the input image objects. They can process multiplerelationships between object and category, such as one-to-one,multiple-to-one, one-to-multiple, and multiple-to-multipleclassification scenarios. Thus, they can effectively output multipleclassification outcomes as multiple trust scores, rather than just asingle classification outcome.

In summary, the image object classification method, system, and tangiblenon-transitory computer readable medium of the present disclosure areprovided for providing an image file including at least one imageobject; performing a process of extracting multiple binary-classifiedcharacteristics on the image object to obtain a plurality of firstresults independent of each other in categories; combining the pluralityof first results in a manner of dimensionality reduction based onconcatenation, and performing a process of characteristics abstractionon the combined first results to obtain a second result; and performinga process of characteristics integration on the plurality of firstresults and the second result in a manner of dot product of matrices toobtain a classification result. Thus, through the aforementioned objectclassification process, it is possible to output implicit informationthat the image object belongs to multiple categories, which isbeneficial to digital documents containing multiple image categories.

Although the present disclosure has been disclosed in preferredembodiments, which are not intended to limit the disclosure, thoseskilled in the art can make various changes and modifications withoutdeparting from the spirit and scope of the disclosure. Therefore, thescope of protection of the present disclosure is defined as definitionsof the scope of the claims.

What is claimed is:
 1. An image object classification method, executedby a processor coupled to a memory, comprising: providing an image fileincluding at least one image object; performing a process of extractingmultiple binary-classified characteristics on the image object to obtaina plurality of first results independent of each other in categories;combining the plurality of first results in a manner of dimensionalityreduction based on concatenation, and performing a process ofcharacteristics abstraction on the combined first results to obtain asecond result; and performing a process of characteristics integrationon the plurality of first results and the second result in a manner ofdot product of matrices to obtain a classification result.
 2. The imageobject classification method as claimed in claim 1, wherein theclassification result comprises the second result, the second resultfurther includes classification reliabilities of the plurality of firstresults, wherein after the classification result is obtained, the methodfurther comprises: documenting the classification result, comprising:selecting at least one of the plurality of first results according to aresult of sorting the classification reliabilities of the plurality offirst results; and recording at least one category attribution name, atleast one object position, and at least one object size corresponding tothe at least one selected first result, in a file.
 3. The image objectclassification method as claimed in claim 2, wherein the object positionincludes a combination of a start-point coordinate and an end-pointcoordinate of the image object, or a combination of a center coordinate,an object length, and an object width of the image object.
 4. The imageobject classification method as claimed in claim 1, wherein after theclassification result is obtained, the method further comprises:performing a graphic-text process on the classification result topresent the classification result in the image file in the form of agraphic block, a text block, or a combination thereof.
 5. The imageobject classification method as claimed in claim 1, wherein the secondresult is obtained by performing the process of characteristicsabstraction on the combined first results in a fully connected manner.6. The image object classification method as claimed in claim 1, whereinthe second result is obtained by performing the process ofcharacteristics abstraction on the combined first results throughmulti-layer perception.
 7. The image object classification method asclaimed in claim 1, wherein the plurality of first results aresequentially concatenated into a combined dimensionality-reduced result.8. The image object classification method as claimed in claim 1, whereinthe multiple binary-classified characteristics are extracted on theimage object from a plurality of image categories that include aplurality of schematic diagrams of characteristics of an electroniccomponent.
 9. An image object classification system, comprising aprocessor coupled to a memory storing at least one instructionconfigured to be executed by the processor to perform a methodcomprising: providing an image file including at least one image object;performing a process of extracting multiple binary-classifiedcharacteristics on the image object to obtain a plurality of firstresults independent of each other in categories; combining the pluralityof first results in a manner of dimensionality reduction based onconcatenation, and performing a process of characteristics abstractionon the combined first results to obtain a second result; and performinga process of characteristics integration on the plurality of firstresults and the second result in a manner of dot product of matrices toobtain a classification result.
 10. A tangible, non-transitory, computerreadable medium, storing instructions that cause a computer to executeoperations comprising: providing an image file including at least oneimage object; performing a process of extracting multiplebinary-classified characteristics on the image object to obtain aplurality of first results independent of each other in categories;combining the plurality of first results in a manner of dimensionalityreduction based on concatenation, and performing a process ofcharacteristics abstraction on the combined first results to obtain asecond result; and performing a process of characteristics integrationon the plurality of first results and the second result in a manner ofdot product of matrices to obtain a classification result.